Emergence of Multi-step Discrete State Transition through Reinforcement Learning with a Recurrent Neural Network
نویسندگان
چکیده
For developing a robot that learns long and complicated action sequences act in the real-world, autonomous learning of multi-step discrete state transition is significant. It is generally thought to be difficult to achieve both holding and transition of states through learning in a recurrent neural network. In this paper, only through the reinforcement learning using rewards and punishments in a simple learning system consisting of a recurrent neural network, it is shown that a multi-step discrete state transition emerged through learning in a continuous stateaction space. It is shown that of the two-switch task, two states transition represented by the two types of hidden nodes emerged through the learning. In addition, it is shown that the contribution of the dynamics in the RNN based on the discrete state transitions leads to repetition of the interesting behavior when no reward is given at the goal.
منابع مشابه
Emergence of Discrete and Abstract State Representation through Reinforcement Learning in a Continuous Input Task
Abstract. “Concept” is a kind of discrete and abstract state representation, and is considered useful for efficient action planning. However, it is supposed to emerge in our brain as a parallel processing and learning system through learning based on a variety of experiences, and so it is difficult to be developed by hand-coding. In this paper, as a previous step of the “concept formation”, it ...
متن کاملMulti-Step-Ahead Prediction of Stock Price Using a New Architecture of Neural Networks
Modelling and forecasting Stock market is a challenging task for economists and engineers since it has a dynamic structure and nonlinear characteristic. This nonlinearity affects the efficiency of the price characteristics. Using an Artificial Neural Network (ANN) is a proper way to model this nonlinearity and it has been used successfully in one-step-ahead and multi-step-ahead prediction of di...
متن کاملNew Reinforcement Learning Using a Chaotic Neural Network for Emergence of "Thinking" - "Exploration" Grows into "Thinking" through Learning -
Expectation for the emergence of higher functions is getting larger in the framework of end-to-end comprehensive reinforcement learning using a recurrent neural network. However, the emergence of “thinking” that is a typical higher function is difficult to realize because “thinking” needs non fixed-point, flow-type attractors with both convergence and transition dynamics. Furthermore, in order ...
متن کاملDiscretization of Series of Communication Signals in Noisy Environment by Reinforcement Learning
Thinking about the “Symbol Grounding Problem” and the brain structure of living things, the author believes that it is the best solution for generating communication in robot-like systems to use a neural network that is trained based on reinforcement learning. As the first step of the research of symbol emergence using neural network, it was examined that parallel analog communication signals a...
متن کاملA model to explain the emergence of reward expectancy neurons using reinforcement learning and neural network
In an experiment of multi-trial task to obtain a reward, reward expectancy neurons, which responded only in the non-reward trials that are necessary to advance toward the reward, have been observed in the anterior cingulate cortex of monkeys. In this paper, to explain the emergence of the reward expectancy neuron in terms of reinforcement learning theory, a model that consists of a recurrent ne...
متن کامل